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TECHNICAL MEMORANDUM 


AN INTELLIGENT POSITION-SPECIFIC TRAINING SYSTEM 
FOR MISSION OPERATIONS 

I. INTRODUCTION 


During a Spacelab mission, approximately 19 payload ground controller positions provide 
operations support for Spacelab experiments. Each ground controller position provides a specific type 
of operations support. For example, the crew interface coordinator (CIC) position manages air-to- 
ground voice communication between the payload crew on the orbiter and the payload operations 
team on the ground. Since each ground controller position provides a specific type of support, ground 
controllers need both generic and position-specific ground controller training. 

Although Marshall Space Flight Center’s (MSFC’s) ground controller training program pro- 
vides very good generic training, position-specific training can be improved by adding position- 
specific training systems. MSFC’s current training program does not contain any position-specific 
training systems. However, ground controllers need position-specific training systems which meet 
all of the following criteria: 

• Provide training on position-specific knowledge 

• Provide training on position-specific subjects and skills one at a time 

• Provide training on integrating position-specific skills and knowledge 

• Provide a training tool that is available 24 hours a day 

• Provide an interactive training environment. 

The purpose of this report is to describe a generic syllabus for position-specific training sys- 
tems, describe a system design for position-specific training systems, describe a process for devel- 
oping position-specific training systems, and describe the MacCIC intelligent position-specific 
training system prototype developed during this research project. The report is arranged as follows: 

• Section n discusses how MSFC conducts ground controller training today and why MSFC 

needs position-specific training systems 

• Section III discusses the objectives for this research project 

• Section IV discusses the approach used to meet the research objectives 

• Section V describes a generic syllabus for position-specific training systems 

• Section VI discusses the range of possible system designs for position-specific training 
systems 



• Section VII describes a process for developing position-specific training systems 

• Section VIII describes the position-specific training system prototype developed during 
this research project 

• Section IX discusses lessons learned during this research project. 


II. BACKGROUND ON MISSION OPERATIONS TRAINING 

MSFC’s ground controller training program provides very good generic ground controller 
training; however, the program only provides a limited amount of position-specific training. MSFC s 
Spacelab ground controller training program is comprised of mission-independent training, mission 
simulations, and line-organization training. 


A. Mission-Independent Training 

Mission-independent training provides an overview of payload operations and does not pro- 
vide position-specific training. Mission-independent training is provided through paper workbooks 
and classroom briefings. Paper workbooks provide general information and address a very broad 
audience. An example of a mission-independent training workbook topic is “Control Center 
Operations.” This workbook provides the trainee with general information about mission manage- 
ment, flight rule changes, and payload operations team responsibilities. The classroom briefings 
provide specific information and also address a very broad audience. An example of a classroom 
briefing topic is “Experiment Power Distribution System.” During this briefing, the trainee is pre- 
sented with an overview of the experiment power distribution system and an overview of how 
experiment power distribution system failures affect payload operations. Although a few of the 
workbooks and classroom briefings describe position-specific responsibilities, mission-independent 
training does not provide position-specific training. 


B. Mission Simulations 

A mission simulation is a practice session for a mission. During a mission simulation, ground 
controllers practice position-specific skills; however, mission simulations are very expensive and 
they do not provide a first-time learning environment. Mission simulations are expensive because 
they require the following resources: 

• Mission Simulation Planning Team 

• Mission Simulation Participants (approximately 200) 

- ground controllers 

- orbiter payload crew (or surrogates) 

- scientists (or surrogates) who have experiments on board the orbiter 

• Facility (Huntsville Operations Support Center) 

- facility costs (computers, power, water, etc.) 

- facility personnel. 
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Mission simulations do not provide a first-time learning environment because ground con- 
trollers are expected to perform position-specific skills and apply position-specific knowledge which 
they already possess. Therefore, although mission simulations provide an opportunity for ground 
controllers to practice position-specific skills, they are not appropriate for training new ground con- 
trollers. 


C. Line-Organization Training 

Ground controllers belong to different organizations within MSFC’s Mission Operations 
Laboratory, and each organization provides some type of line-organization training. Line-organiza- 
tion training does focus on position-specific training; however, each organization provides line- 
organization training in a different way. Some organizations provide line-organization training 
through hands-on exercises, while others provide line-organization training through classroom 
briefings. Although line-organization training provides position-specific training, trainees sometimes 
miss important knowledge or skills for the following reasons: 

• Line-organization training is usually provided verbally 

• Line-organization training is not standardized— training depends on who is doing the 
teaching 

• Line-organization training is not available 24 hours a day or even regularly scheduled 

• Line-organization training is not structured — trainees get pieces of information, not the 
whole picture 

• Line-organization training is affected by personnel changes — people leave the organization 
taking core knowledge with them. 


D. Summary 

Although MSFC's Spacelab ground controller training program provides very good generic 
training, ground controllers gain position-specific knowledge in a very nonstructured way. MSFC’s 
ground controller training program needs a position-specific training system. 


III. RESEARCH OBJECTIVES 

The research project discussed in this report had the following research objectives: 

(1) Utilize techniques in artificial intelligence supplemented by state-of-the-art technology 
in hardware and software to develop an intelligent position-specific training system for the CIC 
Payload Operations Control Center (POCC) cadre position 

(2) Create a generic syllabus for position-specific training systems 

(3) Create and document a process for developing position-specific training systems 

(4) Specify a system design for position-specific training systems. 
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IV. RESEARCH APPROACH 


The project team’s approach to meet the research objectives covered in section III included 
the following two elements: 


• Create a position-specific training system for one of the payload ground controller positions 
(CIC position) 

• Start with an existing development process instead of creating a new development process. 

By creating an intelligent position-specific training system, the project team could evaluate 
all aspects of developing these types of systems. The most important part of this approach was that 
the team could evaluate the entire process, uncover problem areas, and find solutions to the prob- 
lems during the process. The second part of the approach was to adopt an existing development 
process instead of starting from scratch. Since one of the project objectives included using the tech- 
niques of artificial intelligence, the project team used an existing expert system development pro- 
cess Although intelligent training systems are quite different from expert systems, the process to 
develop either type of system is very similar. The expert system-development process adopted was 
found in the book entitled “Knowledge Acquisition: Principles and Guidelines” written by Karen 
McGraw and Karan Harbison-Briggs. This process provided a solid foundation throughout the pro- 
ject. The approach used by the project team to meet the project objectives was very successful. 


V. A GENERIC SYLLABUS FOR POSITION-SPECIFIC TRAINING SYSTEMS 


This section describes the generic syllabus for a position-specific training system. (An 
example of using this generic syllabus for a specific ground controller position-specific training sys- 
tem can be found in section VIII.) The generic syllabus for a position-specific training system 
includes three main components: “Teach me about,” “Practice,” and “Scenarios.’ ‘Teach me 
about” provides a library of position-specific knowledge. “Practice” provides training on position- 
specific knowledge and skills one topic at a time. “Scenarios” provides training on integrating posi- 
tion-specific knowledge and skills. 


A. “Teach Me About” 

The purpose of the “Teach me about” component is to allow the trainee to investigate posi- 
tion-specific knowledge as if they were in a library. ‘Teach me about” should not provide any tutor- 
ing. ‘Teach me about” should contain all the position-specific knowledge the ground controller will 
need to perform his/her job during a mission. As a minimum. Teach me about should contain a 
payload operations overview and a position-specific overview. The payload operations overview 
should contain general information about payload operations onboard the orbiter and on the ground. 
Section VIII discusses this in more detail. The position-specific overview should contain as a mini- 
mum the following elements: 


4 


• Interfaces 

• Inputs/outputs 


• Tools 

• Decision making 

• On-console flow of events 

• Voice protocol 

• Miscellaneous. 


Figure 1 shows the CIC position-specific overview. 

1. I nterfaces - Interfaces should cover all the interaction that this ground controller position 
mil have with other payload operations personnel. This may include othe? ground cmZltoXF 
tions, principle investigators, public affairs, or the orbiter crew. 

. 2 ' f flPUti/Qetpufs . Inputs/outputs should cover all the information that the ground controller 

position receives as well as all information which the ground controller position must produce An 
example of an input is a flight note. An example of an output is a shift report. 

Sion Toni^Sf,H7> 0lS Sh0U !. d the t0 ° ls that ^ ground controIler will utilize during a mis- 

AOS/i^“ reaMime Position-specific software, and the 

controller SS ^ 

. . pa rCcnSQlg Flow Of Events . On-console flow of events should provide a flow chart 

showing what happens from the time the ground controller enters the payload operations control 
^f ^ er t0 begin a shlft untl1 the ground controller hands over to the next shift. The purpose of this 

gr ° Und controllers with some sense of what they can expect Tnd what they 
should be doing. Often new ground controllers feel very awkward because they do not know what m 

h exacUy whi “ lhey "* su PP osed 10 d ° »"- co "sole. Mos. of the time .Ms ™cu?sTecau^ 

of their res^nsib«mes. P ‘ eCeS lnf0rmatlon 10 work w,th and do " ot havc “ overall understanding 

., , 6 ' Protocol . Voice protocol should cover the important concepts about voice Drotocol 

hat every ground controller ts required to know. Since voice protocol is covered ta a Son 
independent training class, only the highlights need to be covered. 

7. Miscellaneous - Miscellaneous should cover any topics that belong in the position- 
spectfic overytew but do not fit in any of the categories listed above. An example of a mStneous 
item would be space classroom or space adaptation syndrome. 

lihrsrv ^ T memion .£! earlier - P^ose of 'Teach me about” is to provide the trainee with a 
ordeMhey 'woultHUte. ^ Sh °“ ld ** free ‘° inves,i S ate the l °P'“ i» “Teach me about” in any 
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Figure 1. MacCIC training system— CIC overview screen. 



B. “Practice” 


The purpose of the “Practice” component is to allow the trainee to practice what they have 
learned one topic at a time. “Practice” contains two major elements. The first element is question 
and answer. The question and answer element should allow the trainee to practice how much he/she 
knows about the topic. The second element should include a variety of unique exercises related to 
the topic. For example, if the training topic is flight notes, there should be several exercises which 
allow the trainee to practice evaluating and responding to a flight note. The combination of question 
and answer and unique exercises should allow the trainee to test how much he/she knows about a 
topic and practice position-specific skills for the specific topic. 


C. “Scenarios” 

The purpose of the “Scenarios” component is to allow the trainee to practice integrating all 
position-specific knowledge and skills at one time. “Scenarios” should strive to simulate the 
decision-making process that occurs during a mission. 


D. Summary 

Intelligent position-specific training systems should contain components for “Teach me 
about,” “Practice,” and “Scenarios.” 


VI. SYSTEM DESIGN CHOICES FOR POSITION-SPECIFIC TRAINING SYSTEMS 

The system design for a position-specific training system could be based on computer- 
assisted instruction (CAI), intelligent computer-assisted instruction (ICAI), or an intelligent tutor- 
ing system (ITS). This section provides a short discussion on each type of system and then explains 
why an ICAI system is the most suitable type of system for the position-specific training systems. 


A. Computer-Assisted Instruction 

CAI has been around since the early 1960’s and basically uses the computer as a tool to 
assist in instruction or training. The goal of CAI was to “build instructional programs that incorpo- 
rate well-prepared course material in lessons that are optimized for each student” (ref. 4, p. 225). 
Barr and Feigenbaum explain that “early programs were either electronic ‘page-turners,’ which 
printed prepared text, or drill-and-practice monitors, which printed problems and responded to the 
student’s solutions using prestored answers and remedial comments” (ref. 4, p. 225). Since CAI is 
based on prestored answers, this type of system does not provide a very rich training environment. 
For example, the training system cannot change the training environment based on the student’s 
responses. However, the advantage of this type of system is that it is very easy to develop. The 
developers can decide on the list of topics and exercises to provide and code the material in a very 
straight-forward manner. 
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B. Intelligent Computer-Assisted Instruction 

Barr and Feigenbaum explain that “in the intelligent CAI (ICAI) programs of the 1970’s, 
course material was represented independently of teaching procedures, so that problems and reme 
dial comments could be generated differently for each student (ref. 4, p. 125). ICAI is a step above 
CAI since it provides the capability to adapt to the student. However, due to this added capability, it 
is somewhat more difficult to develop than a CAI system. ICAI systems eventually evolved into 
what is now referred to as an intelligent tutoring system. In some material, the terms ICAI and ITS 
are still used interchangeably. In this report, a strong distinction is made between an ICAI system 
which incorporates a small amount of intelligence and an ITS which incorporates a very significant 
amount of intelligence. 


C. Intelligent Tutoring Systems 

The jump from an ICAI system to an ITS is really quite extreme. Most ITS ’s are still used in 
university laboratories and have not made it out into industry. This is because they are very com- 
plex, and most of the components of an ITS are still under study. An ITS consists of three main com- 
ponents: an expert module, a student module, and a teacher module. The expert module contains 
knowledge about the subject matter. The student module contains knowledge about the student 
including what the student understands and what the student does not understand. The teacher 
module contains knowledge about teaching, controls the interaction with the student, and evaluates 
the student’s progress. At present, there are many different ideas about how the three modules 
should interact and the specific design of each module. Besides these factors, there are several other 
areas of research in the ITS field such as the environment module and the human-computer interface. 
The environment module is an important area of study and refers to “that part of the system specify- 
ing or supporting the activities that the student does and the methods available to the student to do 
those activities”(ref 5, p. 109). The human-computer interface provides the capability for the user 
and the system to communicate. The human-computer interface for an ITS is inherently complex 
because the users of these systems are by definition working with concepts they do not understand 
well” (ref. 5, p. 143) and, therefore, can greatly enhance or harm the effectiveness of an ITS. In 
general, ITS’s are very complex, very difficult to develop, and need a lot more work before they can 
be mass produced. 


D. Recommendation for Position-Specific Training System Design 

The ICAI system design represents the most economical and robust solution for the position- 
specific training systems. Technology is available to develop an ICAI system in a straight-forward 
manner. The ICAI system will provide better training than a CAI system, but will not require the 
enormous amount of manpower and research necessary to develop an ITS. 


E. Summary 

Although there are several different system designs available for the position-specific train- 
ing systems, the research completed during this project indicates that an ICAI design would be the 
most beneficial. 
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Vn. A GENERIC METHODOLOGY FOR DEVELOPING POSITION-SPECIFIC 

TRAINING SYSTEMS 

The development process presented in this section is a generic development process that can 
be used for developing any position-specific training system. As was discussed in section III, this 
process was adapted from the expert system development process presented by Karen McGraw and 
Karan Harbison-Briggs in their book entitled “Knowledge Acquisition: Principles and Guidelines.” 
Figure 2 shows each phase in the development process. 



Figure 2. Position-specific training system development process. 


A. Define Project Scope 

During phase 1, the project team should define the project scope. The project scope defines 
what the position-specific training system will teach. Since the position-specific training system 
targets a specific ground controller position, the project team members should be people who are 
experts on that specific ground controller position. For example, if the position-specific training 
system will train CIC’s, the project team should consist of people who have flown Spacelab missions 
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as CIC’s. People who are experts in a particular subject, or domain, are called domain experts. 
Therefore, the project team should be made up of domain experts in the particular ground controller 
position domain. The following developmental strategies can be used to define the project scope: 

• Brainstorming sessions 

• Building a concept dictionary 

• Creating a cognitive map. 

1- Brainstor ming Sessions . The first project team meetings should be brainstorming 
sessions. During the brainstorming sessions, the team should identify goals for the position-specific 
training system. The brainstorming sessions should drive out the general requirements for the 
training system. It is likely that the goals discussed during the brainstorming sessions will also 
identify specific hardware and software requirements. For example, suppose the team identifies a 
goal to teach voice communication protocol. This goal requires that the hardware and software 
support the capability to manipulate audio. Therefore, audio manipulation is a requirement on both 
the hardware and software tools needed to develop the position-specific training system. 

2. Building a Concept Dictionary . Following the brainstorming sessions, the project team 
should create a concept dictionary. The concept dictionary “...provides a mechanism to visualize an 
abstraction of the primary concepts in a domain and the terminology used to label them” (ref. 2, pp. 
136-137). The project team should perform the following steps to create the concept dictionary. Each 
team member should: 

(1) Create a list containing every acronym, term, or phrase used in the domain 

(2) Group similar or related terms together 

(3) Label each group of terms. 

A concept dictionary “...is based on the notion that words may be grouped by common 
elements of reference into recognizable concepts” (ref. 2, p. 137). The project team should use the 
concept dictionary as a tool to define the project scope by identifying all the major concepts in the 
specific ground controller position domain. The concept dictionary provides an outline for the project 
scope. The information which follows are excerpts from three different concept dictionaries created by 
three different domain experts during development of the CIC position-specific training system 
prototype. Each of the domain experts grouped these terms together and labeled the terms in a 
category called communications. The domain experts did this individually and did not share 
information. These excerpts show how well the concept dictionary can extract domain concepts. 
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Excerot from Crew Inter face Coordinator Concept Dictionary 


Domain Expert 1 

Domain Expert 2 

Domain Expert 3 

COMM Terminology 

Voice Communications 

Communications 

A/A 

A/A 

A/A 

A/G 

A/G 

A/G 

AOS 

AOS 

AOS 

CH 

COMM 

CH 

FM 

CONF 

COMM 

ICOM 

GDS 

D/L 

KSA 

GN 

DOMSAT 

LOS 

GSTDN 

ECIO 

PM 

H/O 

FOV 

PN 

ICMS 

Hz 

SSA 

INTELSAT 

ICOM 

COMM 

LOS 

LOS 


MILA 

Mbps 


MIPS 

MHz 


NCC 

RF 


SIM 

SCIO 


WCCU 

TV 


WSGT 

uhf 


CH 

VID 


3. Creating a Cognitive Map . After defining the concept dictionary, the project team should 
create a cognitive map. A cognitive map is a domain expert’s ideas concerning primary concepts 
and interrelationships in a domain” (ref. 2, p. 139). Although the concept dictionary provides an 
outline for the project scope, it does not provide enough information to create the training structure 
which is needed to complete the project scope. The training structure is the specific syllabus for the 
position-specific training system. By using the concept dictionary as a building block, the team can 
create the cognitive map which establishes interrelationships between all the concepts in the ground 
controller position domain. Figure 3 shows the cognitive map created for payload operations during 
development of the CIC position-specific training system prototype. Since the project team is 
comprised of domain experts, the cognitive map provides a mental model that can become the 
training structure for the position-specific training system. When the project scope definition is 
complete, the project team can move to the next step in the development process: creating the 
knowledge acquisition plan. 
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Figure 3. Cognitive map for payload operations. 




B. Create Knowledge Acquisition Plan 

After defining the project scope, the project team should write a knowledge acquisition plan. 
The knowledge acquisition plan should contain an entry for every training topic in the position- 
specific training system. The plan should also include a preliminary list of questions and information 
sources for acquiring the knowledge for each topic. The information sources usually range from 
documents to domain experts. The following is an excerpt from the knowledge acquisition plan 
written during this research project: 

Launch/Landing 

What on-board activities are associated with launch and landing? 

Who performs these activities on-board? 

What time do these activities take place? For example stowage, deorbit procedures, etc. 

What ground activities are associated with launch and landing? 

Who performs these activities on the ground? 

Informati on Source : Domain experts and training materials. 

There are two types of knowledge that need to be collected for a position-specific training 
system. The first type is the domain knowledge and the second type is pedagogical knowledge. For 
example, the CIC position-specific training system prototype provides training on flight notes. 
Therefore, the team had to collect domain knowledge about flight notes and domain knowledge about 
training new CIC’s on handling flight notes. The knowledge acquisition plan should address both 
types of knowledge. The project team should use the knowledge acquisition plan as a guideline for 
collecting all the domain knowledge and pedagogical knowledge necessary for the position-specific 
training system. 


C. Conduct Domain Expert Orientation Meeting 

After creating the knowledge acquisition plan, the project team lead should conduct a domain 
expert orientation meeting. This meeting introduces new domain experts to the project goals and 
activities. Although the project team is made up of domain experts, there will usually be other 
domain experts who participate in the development process. The domain expert orientation meeting 
addresses the new domain experts. The purpose of this meeting is to help domain experts 
understand what the team is attempting to build, what their role in the program will be, and what 
they can expect from knowledge acquisition’ ” (ref. 2, pp. 88—89). The following is an example of 
some of the topics addressed during the domain expert orientation meeting conducted during this 
research project: 

• Introduction to artificial intelligence 

• Introduction to intelligent training systems 
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• Functional goals of the CIC position-specific training system 

• Knowledge acquisition for the CIC position-specific training system. 


D. Perform Knowledge Acquisition 

After the domain expert orientation meeting, the project team should begin conducting 
knowledge acquisition sessions. Knowledge acquisition is the process of extracting, transforming, 
and transferring expertise from a knowledge source...” (ref 2, p. 344). At this stage in the project, 
the project team personnel usually expands. The original team members continue to serve as domain 
experts and new members join the project team to serve as knowledge engineers. A knowledge 
engineer is an individual who ”... works with domain experts to acquire, structure, and translate 
domain knowledge...” (ref. 2, p. 345). For each knowledge acquisition session, the knowledge 
engineer should perform the following steps: 

• Pre-knowledge acquisition activity 

• Knowledge acquisition session 

• Post-knowledge acquisition activity. 

As was discussed above, there are two types of knowledge that need to be collected for a 
position-specific training system. The knowledge acquisition sessions are used for collecting both 
types of knowledge. 

1. Pre-Knowleflgp. Acquisition Activity . The knowledge engineer should prepare for a 
knowledge acquisition session by contacting the domain expert to schedule a session, researching 
the session topic, and preparing a pre-knowledge acquisition session form. The pre-knowledge 
acquisition session form should include as a minimum the date, place, time, and purpose of the 
knowledge acquisition session. The knowledge engineer should also list preliminary questions for 
the domain expert to begin thinking about before the session. Both the knowledge engineer and the 
domain expert use the information in the pre-knowledge acquisition session form to prepare for the 
knowledge acquisition session. 

2. Knowledge. Acquisition Session . There are many different techniques that can be used to 
perform knowledge acquisition. Karen McGraw and Karan Harbi son -Briggs^ provide an excellent 
discussion on using the interview as a knowledge acquisition technique. During this research project, 
the knowledge engineers used several knowledge acquisition techniques including. 

• Reviewing available documentation 

• Conducting interviews with domain experts 

• Communicating with domain experts through electronic mail 

• Observing mission simulations. 
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However, since most of the knowledge ground controllers possess is obtained through on- 
console experience, interviews are usually the most beneficial. Many times interviews are the only 
appropriate technique available to acquire certain pieces of knowledge. Therefore, the knowledge 
engineering team should be prepared to spend a significant amount of time interviewing domain 
experts. The knowledge acquisition sessions should be recorded on video or audio tape in order to 
store all the knowledge discussed during the session. The knowledge engineers should utilize these 
tapes when they prepare the post-knowledge acquisition session notes. Knowledge acquisition 
sessions conducted as an interview with a domain expert should not last longer than 2 hours. The 
knowledge engineering team found that interviews which lasted longer than 2 hours tended to lose 
focus. 

3- Post-Knowledge Acquisition Activity . After conducting a knowledge acquisition session, 
the knowledge engineer should perform the following steps: 

• Document the session results on a post-knowledge acquisition session form 

• Send the results to the domain expert for review 

• Incorporate any comments or modifications made by the domain expert 

• Submit the post-knowledge acquisition form to the project team. 

When the project team receives a post-knowledge acquisition form, the team should 
determine if the knowledge acquisition session goals were met and whether further sessions will be 
necessary. The project team must be comprised of domain experts in order to make these 
determinations. 


E. Design and Develop System 

Once the team has collected all the domain knowledge for the position-specific training 
system, the team can move into the design and develop system phase. The manner in which the 
team performs this phase depends on which system design is selected. Section VI describes a range 
of system designs which are applicable to a position-specific training system. The information which 
follows is based on the research conducted during this research project. Therefore, this information 
should be carefully evaluated before a team decides this is the best route to follow. If the team 
decides to adopt the type of system design recommended in this report, the following steps should 
be performed during this phase: 

• Layout and detail the overall design concept 

• Design and develop the user interface for each topic 

• Design and develop the knowledge base for each topic 

• Connect the user interface and the knowledge base. 
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1 l ayout and Detail the Overall De sign Concept. The very first step the team should perform 
in the design and develop system phase is to layout and detail the overall design concept. The overall 
design concept should be based on the generic syllabus presented in section V of this report and the 
cognitive maps which were created when the team defined the project scope. The generic syllabus 
presented in section V provides the foundation for the layout of the system s user interface. The team 
can start with this layout and then modify the parts which will be specific to the particular position- 
specific training system they are developing. For example, figure 1 shows the layout for the CIC 
overview screen which is the position-specific overview screen in “Teach me about. If the team is 
building a system for the data management coordinator (DMC) position, they can begin by using the 
CIC overview screen as the starting point, and then modify it so that it contains information specific 
to the DMC position. Once modified the screen can be used as the DMC overview screen. During this 
part of the design and develop phase, the team should evaluate each of the three areas of the generic 
syllabus and define the specific information each area should contain based on the specific ground 
controller position. As soon as the team completes the layout and detail of the overall design concept 
step, they can then move on to design and development tasks for each of the individual topics. 

2 . Design and Develop the User Interfac e, for Each Topic. The first step in the design and 
develop phase for a specific topic focuses on the user interface design. The team should use the 
process shown in figure 4 to design and develop the user interface. First, one of the team members 
should create a user interface design for the topic. The design should be based on the preliminary 
user interface design concepts created during the definition of the overall design concept. The devel- 
opment team member should submit the design to the project team. Once the project team approves 
the design, the developer should create a software prototype of the design. Once the prototype is 
complete the team should ask potential users to evaluate the design. The users should include both 
experienced ground controllers and ground controller trainees. The project team and users should 
work together to revise the design. After incorporating the revisions, the developer should create the 

final version. 


3 Design and Develon the Knowledge Bas e for Each Topic. During this step in the design 
and develop phase, the project team should transform the knowledge collected in the knowledge 
acquisition sessions into facts and rules. These facts and rules, which represent the domain knowl- 
edge are then stored in the knowledge base. This step in the design and develop phase will be 
guided by the overall system design defined during the layout and detail for the overall design con- 
cept. 

4 Connect the User Interface and the Knowledge Bass- During the last step in the design 
and develop phase for a particular topic, the developer connects the user interface and the knowledge 
base This step is really a clean-up step since the developer has usually added bits and pieces ol 
code to connect the user interface and knowledge base throughout the two previous steps. Although 
the previous steps were presented as individual steps, they are really very tightly integrated. 


F. Summary 


The methodology presented in this section can be used as a model to create other ground 
controller position-specific training systems. Steps 1 through 4 can be applied to any type of 
position-specific training system which is developed. As was stated earlier, step 5 may need to be 
modified based on the type of system design which is selected for the position-specific training 
system. Although each position-specific training system will contain different domain knowledge, the 
process to develop a position-specific training system is generic. 
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Figure 4. User interface design process. 


VIII. MacCIC : AN INTELLIGENT POSITION-SPECIFIC TRAINING 

SYSTEM PROTOTYPE 


MacCIC is an intelligent position-specific training system prototype for the CIC ground 
controller position. MacCIC has been under development throughout the course of this research 
project. A significant amount of the development time has been spent in the knowledge acquisition 
phase. MacCIC contains hooks for each major component that should be part of a position-specific 
training system. However, MacCIC is a prototype and will require a significant amount of work to 
make it a complete training system. MacCIC’s training structure is based on the generic syllabus 
presented in section V of this report. Although MacCIC provides training for a specific ground 
controller position, many of MacCIC’s elements are applicable to other ground controller position- 
specific training systems. Upon completion, MacCIC will meet all of the following criteria: 
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• Provide training on CIC specific knowledge 

• Provide training on CIC specific knowledge and skills one topic at a time 

• Provide training on integrating CIC specific knowledge and skills 

• Provide a training tool that is available 24 hours a day 

• Provide an interactive training environment. 


A. MacCIC Training Structure 

The MacCIC training structure has three major components: ‘Teach me about,” “Practice,” 
and “Scenarios.” ‘Teach me about” provides a library of CIC specific knowledge. “Practice” 
provides training on CIC specific knowledge and skills one topic at a time. “Scenarios” provides 
training on integrating CIC specific knowledge and skills. 

1 Tpach Me About . “Teach me about” provides a library of information on a variety of 
topics. The trainee is free to investigate the information in any order and in any amount of detail. 
‘Teach me about” provides information on the following topics: 

• Payload operations overview 

• Network communications 

• Mission milestones 

• Documentation 

• CIC overview. 

a. Pavload Operations Overview . The payload operations overview provides information 
about activities, personnel, and facilities on the ground and onboard the orbiter during a spacelab 
mission. Most of the information provided in the payload operations overview is also provided in 
mission-independent training. However, MacCIC provides a concise presentation of the information 
that is pertinent to the CIC position. 

b. Network Communications . Network communications provides information on ground and 
space network communications. CIC’s need to understand this information in case network 
communications fail. The information presented specifically addresses types of failures that affect the 
CIC position. For example, network communications provides the trainee with information on 
tracking data relay satellite system (TDRSS) failures and provides information on how CIC’s 
respond to TDRSS failures. 

c. Mission Milestones . Mission milestones provides information on all the major pre- 
mission events that occur during the 3 years prior to a spacelab mission launch. These mission 
milestones include events such as the integrated payload design reviews, crew training, payload 
operations team training, and joint integrated simulations. Mission milestones includes a 
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cartoon-story entitled “A day in the life of a principle investigator.” The cartoon-story provides a 
step-by-step look at an entire Spacelab mission. The story begins with the principle investigator’s 
proposal to NASA to fly an experiment onboard the space shuttle. The story ends with the process 
of distributing the mission science data after the orbiter lands. The mission milestone information 
provides a general introduction to the Spacelab program. 

d. Documentation . Documentation provides information on all the mission documents a CIC 
should be familiar with. It includes a description of each document and the document’s table of 
contents. When MacCIC displays the table of contents, it highlights the topics which are most 
important to the CIC. The mission documents addressed in the documentation section are as follows: 

• Basic Flight Definition Document 

• Data Flow and Data System Configuration 

• Flight Definition Document 

• Orbital Mechanics Analysis — Appendix A 

• Generic Joint Operations Interface Procedures for Spacelab 

• Integrated Training Plan 

• Operations Control Handbook 

• Operations and Integration Agreement 

• Payload Crew Activity Plan 

• Payload Flight Data File 

• Payload Operations Checklist 

• Payload Operations Control Center Configuration Requirements Document 

• Payload Operations Control Center Integrated Requirements Document 

• Payload Operations Guidelines (All Flights) 

• Payload Operations Guidelines (Flight Supplement) 

• Payload Operations Handbook (All Flights) 

• Payload Operations Handbook (Right Supplement) 

• Space Shuttle Operational Flight Rules Annex 

• Spacelab Verification Plan 
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• Space Transportation System Flight Data File 

• Training Annex No. 7. 

e. CIC Overview . The CIC overview presents a table of contents for all the CIC specific 
knowledge. It is the most important item in “Teach me about.” New CIC’s will spend most of their 
time studying the information provided through the CIC overview. Figure 1 shows the wide range of 
subjects addressed in the CIC overview. The CIC overview structure is generic and applies to all 
ground controller positions. For example, all ground controllers interface with other personnel, accept 
inputs, generate outputs, use position-specific tools, make decisions, and use voice protocol. 
Therefore, the CIC overview section is an example of a generic structure that can be used to 
represent any ground controller position environment. 

2. Practice . “Practice” allows the trainee to practice CIC specific knowledge and skills one 
topic at a time. For each topic, MacCIC provides both question and answer exercises and unique 
exercises which relate to the specific topic. During the practice exercises, MacCIC provides tutoring 
when necessary. “Practice” will eventually contain exercises for the following topics: 

• AOS/LOS clock and groundtrack display 

• Communications panel 

• Console operations 

• Crew procedures 

• Enabling/disabling principle investigators on air to ground 

• Flight notes 

• Interfaces 

• Joint operations procedures 

• Making decisions 

• Log book 

• Operations change requests 

• Payload crew activity plan 

• Payload operations handbook procedures 

• Real-time displays 

• Shift handover 

• Shift report 


20 



• TAG’s/TPR’s 


• Uplink summary 

• Voice protocol. 

At this time “Practice” contains one fully developed topic and partial entries for approxi- 
mately half the others. The enabling/disabling principle investigators on air to ground topic has been 
completed. The other topics will be completed during the next year. 

o scenarios. “Scenarios” provides a way for the trainee to practice making decisions which 
require integrated knowledge and skills. In a real mission environment, each ground controller mus 
use all the domain knowledge and skills he or she possesses to make many important decisions^ 
These decisions almost always require knowledge about many 

“Scenarios” section is to provide an environment which allows the trainee to practice maxing 
dedstons using integrated skills and knowledge. Originally, the development team had planned to 
maters com g pone„ g , a a mini.simu.ation, Howlver, the technology that would 1 e requued to emulate 
a simulation, as well as the complexity involved in developing a mini-simulator, did not allow the 
team to Dursue this direction. Therefore, after much consideration, the team decided on the 
“Scenarios” component which tests the trainees capability to use integrated knowiedge and skills to 
make decisions. At this time, the team has written many scenarios and plans to develop the 
software to support them during the next year. 


B. MacCIC System Design 


MacCIC’s system design utilizes artificial intelligence which is “...the part of computer 
science concerned with designing intelligent computer systems, that is, systems that exhibit the 
characteristics we associate with intelligence in human behavior— understanding language learning, 
reasoning solving problems, and so on” (ref. 1, p. 2). The MacCIC system design provides a generic 
structure^ that can § be used as a model for creating other ground controller position-specific training 
systems. As shown in figure 5, the MacCIC system design consists of three primary components, 
the user interface, the inference engine, and the knowledge base. 


1 User Interface . The user interface accepts user input and presents system output to the 
trainee. MacCIC’s user interface is a graphical user interface (GUI) whichprovidesaway for the 
trainee to issue commands to the computer by using an input device (su f f , a u _ 

maninulate obiects on the screen. Some of the common objects found in a GUI are windows^ pun 
down menus, ind icons. MacCIC’s user interface complies with Apple’s Human Interface Guidelines 

graphical user interface style guide. 


2. Inference Engine , 
evaluate the trainee’s inputs 
inference engine is provided 
configured. 


The inference engine uses the knowledge in the knowledge baseto 
and to make decisions about providing feedback to the trainee. The 
by a commercial software product; however, the inference engine can 


be 
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Figure 5. MacCIC system design (source: adapted from reference 3, p. 34). 

3. K nowledge Paso - The MacCIC knowledge base contains the following types of knowledge: 
• Knowledge about the CIC position 


• The trainee’s knowledge about the CIC position 
Knowledge to evaluate the trainee’s interactions with the MacCIC system. 

• . . The knowledge base contains the knowledge listed above in the form of facts and rules. A fact 

pmimH ex ample of a fact is: a CIC can enable a principle investigator on air to 

ground A rule is usually in the form of an if-then statement about facts. An example of a rule is- if 
the air-to-ground communication status is not good, then a CIC should not enable a principle 
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framework for creating the MacCIC system design. 


C. MacCIC Hardware and Software Components 


The MacCIC training system consists of the following hardware 


and software components: 


Hardware : 

. Apple Macintosh Ilci with extended keyboard and mouse 
- 8 MB memory and 80 MB disk space 

• RasterOps 19-in color monitor 

• RasterOps 24XLTV video card 

• Pioneer laserdisc player 


• Pioneer VHS VCR 

• Apple CD-ROM player 

• 9-in television monitor. 

The MacCIC training system hasthree^ 

Ilci, keyboard, and RasterOps monitor. Since CIC s us ^ t exercises. The Spacelab 

training system uses Spacelab mission video during video of the 

mission video includes a shuttle launch an an 1 g. > P ' y MacCIC hardware components which 
orbiter payload crcwperfo™^^ Raster0ps P 24 XLTV video 

provide this capability are. the VHS P often need t0 motivate trainees in order to 

S gg-W p— a variety - sound ejects 

and music through digitized audio and an Apple CD-ROM player. 


Software : 

• Silicon Beach’s Supercard 
. Neuron Data’s NEXPERT OBJECT. 

As mentioned in the system design 

components: the user interface, the inference en©i ^ , in f erence engine and the software 

software. 
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D. Summary 

as a modd ^cZ^'oZ ^ ^ ^ ““ ™ <* — 

major training structure components MarPTr n ™ h °° speciflc trainin g systems. With its three 
conquer environmen i for re P resenlin S «■» ground 

ground controller positions. For instance, although the^Mcinc skills f^nuf* d ' ffcrences amon S 
from those of the CIC position all nositinn-«rw;fi,> r. o Specihc skl,ls for a D MC position will differ 
practice sessions on position-specific skills. ming systems need t0 provide training and 

training systems. If Y the ICAf system design is^sed 8 & "h** 61 f ° r creatin 2 other Position-specific 
a user interface, an inference engine and a knowledge ach position-specific training system will need 
MacCIC’s hardware and software ^cimnonenJ l d T ain knowIed S e - Although 

training systems, each system iZ'T ^ 

system developer should consider the overal? svstem r^n • S ' Therefore ’ a Position-specific training 
deciding what type of hart^^XS cC„S‘ t ” a " d ^ ttCh " 010 ® 


IX. LESSONS LEARNED 


projects, the fessons learned are ^mporun t ^fnce^he v *™ P ? rLant ^ ssons learned - Like many research 
failure of continuing work in this area. The lessons learned ’were^Iowr' °" ‘ he SUC “ SS ” 

(1> hw™faU XPerL< mUS ‘ 1,6 aSSi *" ed ‘° ,he P r °J ecI a "« teir time dedicated to the project or 

system teig^Tlectfd *“!"«. adless of the type of 

that knowledge. At lei. sCSo" UteTnow ZZ t-T ?"“? ? d0mai " Provide 

not wntten down anywhere. Therefore if manaeemfnt^ 10 ^ S P acelat> ground controllers possess is 
experts will participate in the development nrnJP?!? 1 does guarantee that several domain 
ground controllers Se al f n0th ,: n8 Spacelab 

organized around the principle oSL S a m?.S' T herefore ’ lhe Project should not be 

project, no domain experts were officially assigned mfe DXrThemf 0Vlde ,il !n0Wled8e ' D “ rin * this 
dependent on borrowing what little time *? P T Ct WaS 

project survived was because the nroiect lead hann^H k don l a n ex P erts - The only reason the 

a: 

01 sssa: sss." ax- as zszsssz •— 

knowledge engineers°each had^a dtfferenTran! ?e^ ^!f d^ 0 &S knowIedge en g ineer s. The 

knew a little and some knew a lot The knowledge enlw expdrtl f • Some knowledge engineers 

had a very hard time determinmg when they ha/co„eLd eimgh “wTSgeitm a ropi!^ 
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S more Sedge £,uiSt£ons were 
required. This process worked out quite well. 

(3) The knowledge acquisition plan should contain both domain knowledge and pedagogical 
knowledge. 

subject^e^owfed^^qu^itim? pl^shouW ^s^ad^^Thow^he^dag^ic^^o^vledgelfor the 
subject will be acquired. 

(4) The technology required to develop a simulation capability is not yet available. 

The original proposal for this research project discussed the desire to create sotnelypeof 
stand-alone simulation capability However during the this 

mwmssmsf 

objective has been met. 

(5) Interviews with domain experts were identified to be the best knowledge acquisition 
technique available for the payload operations domain. 

ttttxsxsxxx SSKSST.2K «**- t 

position-specific training system. 


X. CONCLUSIONS 


Since payload ground controllers acquire position-specific training in » nonstrocmred way, 

ISnXvevS 

and insures that they are available throughout ^ devemp^m proce^. faSrettta* 

the'r^t'of fte’^^timi-s^^c tr^h^^ystems. By using intelligent ^ifem-s^ifa^t rajnin^ 
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capability to store domain knowledge which is often lost due to personnel changes Allhoueh this 

r PaC ,n ab c gr ° Un c d C0ntr0 " er these recommendatof 

Si^g program SpaCe Sta,,on Freedom <S S ' Fmdom > P^oad ground controller 


XI. RECOMMENDATIONS 


Based on the research presented in this report, the following actions are recommended: 
maindiie’trSZg prog ft rL MaCCIC p0sWon - speciflc tra,nin 8 s )™tem ""d 'hen move MacCIC into the 

training < ^ol PrePare ” evaluation Procedure to test MacCIC’s effectiveness as a position-specific 

nt . (3) -[ nvesti gate the possibility of creating similar position-specific training systems for the 
other ground controller training positions. This will depend on the availability of domain experts to 
participate in the development process. y experts t0 

(4) Once the S.S. Freedom payload operations concept is well-defined and experienced 

s^creSgTysremf' 6 ' “ ^ ” Sh< ’ Uld ‘-loping "2- 
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